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Abstract 

A dynamic trustiness authentication framework 
based on the integrity of software's behavior is 
proposed in this paper. The method to extract SIBDS 
(Software Intended Behaviors Describing Sets) and 
SB AC (Software Behavior Authentication Code) from 
the binary executable is introduced. In the framework, 
when the software begin to run, it should be monitored 
by SBMC (Software Behavior Monitoring Center), then 
the real API function invoking sequence will be 
acquired. The framework uses the software behavior 
comparison algorithm to verify whether the API 
invoking sequence gotten from the actual behavior is in 
accordance with SB AC; thereby the software's 
dynamic trustiness can be detected and guaranteed. 
The experiment results demonstrate the efficacy of the 
dynamic trustiness authentication framework. 

Keywords: Function invoking, dynamic trustiness, 
behavior trustiness, behavior integrity 

1. Introduction 

Dynamic trustiness is indicated through 
comprehensive and deep understanding of the 
unconsistency between the intended software's 
behavior and the practical behavior. To ensure the 
dynamic trustiness is to make sme that the entity's 
behavior is always works in the intended way and goes 
towards the intended du^ection. The dynamic 
trustiness's related theory and technology, based on the 
software behavior, is a generally acknowledged 
fimdamental problem with great difficulty and also a 
significant problem that must be solved. 

Some domestic enterprises, university and related 
research institutes have done a lot study and research 
actively in the field of tmsted computing ^'"^l 



Meanwhile, a series of practical work focused in 
software behavior intrusion detection has been done in 
the field of host intrusion detection. In 1996, Stephanie 
Forrest and his colleagues proposed a model for 
detecting intrusions at the level of privileged process 
using sequences of system calls: N-gram model, which 
is a typical representative of intrusion detection 
techniques based on processes' behavior ^'l 
Subsequently, inspired by Forrest, more outstanding 
models and achievements have spring up '''""I 

CuiTently, large quantities of research achievements 
have been accumulated in the field of software static 
analysis and intrusion detection both here and abroad. 
But in the field of trusted computing, the research on 
the software dynamic trustiness authentication model 
continues to be scarce. 

In this paper, we put focus on the research of the 
software behavior and proposed a framework of 
dynamic software's tnistiness authentication, based on 
the software behavior integrity. 

2.Description of the Dynamic Trustiness 
Authentication Framework 

2.1 General Framework 

To introduce the framework we proposed, we make 
the following definitions. 

Definition 1: Software Behavior. Software 
Behavior are any changes, influences or any operations 
made to the other independent entities when the 
software works as an independent entity. 

Definition 2: Software Behavior Integrity. 
Software Behavior Integrity means the consistency 
between the intended behavior of the software and the 
real behavior in the real executing process of the 
software. 
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Figure 1. Dynamic Trustiness Authentication 
Model Based on Software's Behavior Integrity 

In this paper, we consider that dynamic trustiness 
means the software works in the intended way, say the 
software behavior integiity is not broken. Software has 
dynamic state in the process of execution, so that the 
software is trusted at some time does not means it is 
trusted in the process of execution all tlie time. For the 
sake of this, the authentication procedure is sustaining 
and accompanying with the real executing process of 
the program. 

To make the dynamic trustiness authentication more 
effectively, we propose the dynamic trustiness 
authentication framework on the basis of the software 
integrity, as is shown in Figure 1. 

Firstly, we do the static analysis on the program's 
binary code to get the executing flow graph so that we 
can extract the software's intended behavior describing 
set (we describe the behavior using API fimctions 
invoking in this paper) from the software's intended 
behavior. Then, we simplify the software intended 
behavior describing set and store the set in the way we 
defined and get Hhe SBAC (Software Behavior 
Authentication Code), which can be released along 
with the software or can be released by some 
authoritative websites. 

When the software is in real executing process, 
SBMC (Software Behavior Monitoring Center) 
monitors the real executing process and get the trace of 
the software's execution (we describe it using API 
functions invoking sequence in this paper). Then, 
DTAC (Dynamic Trustiness Authentication Center) 
will match the real API function invoking sequence 
with the SBAC to check the software behavior 
integrity. If the behavior integrity is not broken, the 



current state of the software is trusted, or the current 
state is not trusted and SBCC (Software Behavior 
Control Center) must intervene and take corresponding 
measure to control the unknown behavior. 

2.2 Theory of the Software Behavior's 
Trustiness Authentication 

The software state k is influenced by the module 
behavior M, and whether or not the trusted state k can 
conduct the trust transitive depends on trustiness of the 
module behavior M that drives the state transition. In 
the proving process of the software behavior (process) 
trustiness, the intended behavior is represented by the 
trustiness behavior intended policies of the interpellant 
(DTAC). 

Any process module that is inconsistent with the 
intended behavior policy will be regarded as 
unauthentic behavior, say the corresponding software 
process state is regarded as trustless, so lhal Ihe 
discrimination process of the soflwarc behavior's 
trustiness can be converted into the trustiness 
verification of the software module' behavior. 

One time behavior of a specific module M of the 
software is trusted, means the behavior of the module 
M is consistent with the trusted behavior intended 
policy of the DTAC. Trusted behavior intended policy 
consists two parts: the intended behavior of the module 
M, and the jumping relations of the module M and the 
forward and backward modules. 

In this paper, the trusted behavior intended policy is 
to mean the software's intended behavior describing 
sets or the Software Behavior Authentication Code. 

As is presented before, the software behavior's 
trusted state is influenced by the module behavior. In 
the paper, we use the transfer fimction f(ki ,Mi)=ki+i to 
express the module software Mi makes the software 
trusted state transfer from ki to ki+i. The trusted module 
behavior will not change the creditability of the 
software behavior, based on which we make the 
definitions. 

Definition 3: If the software behavior state ki and 
the module behavior Mi is trusted, meanwhile f(ki, 
Mi)= ki+i, then the software behavior state ki+i is 
tmsted. 

According to the definition 3, we can conclude the 
theorem I. 

Theorem 1: At a given time t, if the software state 
kt is trusted, and every software behavior module Mi (1 
^ i ^ n) of the software module behavior sequence 
M=MiM2 ■■■ Mn is trusted, then the software state 
k,+i(=f(kt,M)) is trusted, after the occurrence of the 
module behavior M 

The significance of the theorem 1 is quite important. 
Firstly, the state space of the software is infinite, which 
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makes it hard to distinguish the trustiness of the 
software state. But, on the base of the theorem 1, at a 
given time, we can verify the trustiness of the software 
state, and the afterward module behavior to solve the 
problem of judging any software states' trustiness. 
Secondly, theorem 1 helps to transform the 
measurement and verification of the software behavior 
state to the module behavior, which provide the 
practical process of software behavior tmstiness 
authentication with well theory basis. 

It is known, according to the analysis, that the 
software behavior state's authenticating process can be 
described by the detennined finite state automata 
(DFA). For software in the process of execution, it 
contains an initial state and one or more terminal 
states. We regard the trusted module set as the input 
alphabet set (conversion condition), and every trusted 
module's input will change the cmrent state to another 
new state. On the basis of the theorem 1, we can 
represent the software behavior trustiness 
authentication framework using DFA as follows: 

The according five tuples of the DFA is M=(K, E 
,f,S,Z): 

1) K is a finite set containing multiple kinds of the 
software's trusted state; each element in K is a 

tnisted state. 

2) Y. is a I'niilc input alphabet set; each element in it 
is regarded as an input symbol (it is the module 
behavior Mi in this paper). 

3) f is the transfer fimction, which is the mapping 
relation of the KX E ^K. For example, if f(ki, 
Mi)=kj, (ki e K, kj e K), it means that the current 
trusted state is ki , when the input symbol is the 
API fimction invoking sequence module Mi, the 
state of the software transfers to the next state kj, 
and we consider the kj as the subsequent trusted 
state of ki. 

4) S £ K is the only one initial state. In this paper, it 
means the software's initial trusted state, which is 
unique. 

5) Z is the set of the terminal states, which indicates 
the software's termination state and it can be more 
than one state. 

DFA M=(K, E ,f,S,Z) and its behavior can expressed as 

follows: 

K:=S; 

M:=getModule; 
While BeTrusted(M) do 
{ 

K:=f(K,M); 

if Ke Z then return ('yes') 
M:= getModule; 

}; 

return ('no') 



2.3 Implementation Method of Software 
Behavior' Dynamic Trustiness Authentication 
Framework 

2.3.1 Process of the Software Dynamic Trustiness 
Authentication. We describe the software intended 
behavior based on the invoking sequences of the Win32 
API, so we define SIBDS and AM as below. 

Definition 4: SIBDS. SIBDS (Software Intended 
Behaviors Describing Sets) is an anticipated behavior 
set which describes the normal API invoking 
sequences and its interrelations. It is composed of API 
functions Module Set (AMS) and Modules 
Relationship Set (MRS), say SIBDS=<AMS, MRS>. 

Definition 5: AM. AM (API functions invoking 
Module) refers to an API functions module which has 
a relative stable and independent sequence of API 
functions invoking when the software runs normally. 




Figure 2. Call graph of AM of sample program 

In this paper, we divide all the API invoking 
sequences into many independent API invoking 
sequence module according to the program flow and 
represent the relations and the content of all the 
modules using SIBDS. In this way, AMS and MRS 
can present the software behavior with the API 
function invoking sequence module in different levels. 
Figure 2 shows an example of our sample program (Mi 
is AM). 

After obtaining AMS and MRS of the software, on 
the basis of the trustiness authentication theory 
introduced in section 2.2, we can build a corresponding 
finite state graph to describe the whole process of the 
software's trustiness authentication. 

Figure 3 is the finite state graph built on the basis of 
representing the software's trustiness authentication 
process using determined finite state automata. It 
describes the software's trustiness authentication 
process from the level of API fimction invoking 
sequential module. 
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Figure 3. Corresponding finite state graph of 
Figure 2 

Further more, we can define and describe each AM 
to describe tlie software behavior completely. With a 
moderate match algorithm, we can do the module 
behavior tmstiness authentication to AM and form a 
complete authentication system and framework. 

2.3.2 Extraction and Optimization of AM. When 
extracting the softwM-e intended behavior, we firstly do 
static disassemble to the software's binary executable. 

Accordrag to the disassemble code, we can insert 
"Module BookMark"(BM) into the following places: 
(i) the start address of the current flinction;(ii)the 
address wlicrc llie jump instructions jump to; (iii) the 
next instruction's addi'ess of the jump instructions; (iv) 
the address of return instniction. 

After inserting some Module Bookmarks (BM) into 
the assembled code, we can divide the code into 
several SIPs (Sequential Instruction Pieces), and all of 
these SIPs form SIPS (Sequential Instruction Pieces 
Sets). 




Figure 4. Process of dividing assemble code 
into SIPS 



Then, we give each SIP an unique label to identify 
the SIP and record the jumping relationship between 
different SIP, say SIPRS (Sequential Instmction Pieces 
Relationship Sets), using directed edge (each directed 
edge represents a jumping relation in the call graph), 
draw the flow graph according to the SIPS and SIPRS 
and finally we have a call graph of a AM. In the same 
way, we can draw other AM's call graph accordingly. 
The drawing process is shown below in Figure 4. 

In the process of extracting the sequence of API 
invoking, we focus on the functions (self defined 
functions and API functions) invoking instructions and 
all kinds of jumping instructions (e.g., IMP, JXX, 
LOOP, RET and so on) and neglecting aU the other 
instructions. In this way, the graph we finally get is an 
API invoking relationship graph. Another extraction 
method is proposed in the literature. ^'^^ 

2.3.3 Storage Structure of the API function 
invoking module (AM). MRS describes the trusted 
relations between AMs at a high level. To ensure that 
the software is trusted at the level of API fimction, it is 
necessary to ensure the trustraess of each AM. 

In the MRS's FSM (Finite State Machine), AM is 
the input element of the finite input alphabet. At the 
time of the state conversion of the FSM, the trustiness 
authentication of the input element is quite essential, 
which makes it necessary to define an efficient storage 
stmcture and provide a complete authentication 
method of AM. 

Based on software's API function invoking flow 
graph, we can make the definition of the AMS. 

The hierarchical relationship among the data 
structures of SIBDS is described as follows; 

I — SIBDS (Software Intended Behaviors Describing Sets) 
I— AMRS (API Modules Relationship Set) 
I— AMS (API Module Set) 
I — AM (API fiinctions invoking module) 
I— SIPRS (SIP Relationship Sets) 
I — SIPS (Sequential histmction Pieces Sets) 
I — SIP (Sequential Instniction Pieces) 
I— APIFC (API Fimctions Calling) 
I— SDFC (Self-Defined Fimctions Calling) 

Limited to the page cover, we won't give the related 
storage data structure in detail in this paper. 

2.4 Software Behavior Monitoring Method and 
Software Trustiness Authentication Algorithm 

In this paper, we monitor the real execution process 
of the software to get a real-time API functions 
invoking sequence, comparing which with the 
Software Behavior Authentication Code to realize the 
tmstiness authentication and control of the software 
behavior. 
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The software behavior authentication algorithm, in 
essence, is a matching algorithm which matches the 
real API fimctions invoking sequence with the 
Software Behavior Authentication Code (storage 
definition of the SIBDS). 

3. Related Experimental Data and Result 
Evaluation 

Overflowing attack and malicious remote thread 
injection are two typical attack techniques to deviate 
the software's dynamic behavior. Overflowing attack is 

the main skills used by hackers to conduct the remote 
attack, and remote thread injection is a common way 
used by local host's vicious code to hide itself from 
being detected. To test the dynamic trustiness 
authentication's validity of the framework we proposed 
in this paper, we carry out three typical experiments; 
the first one is the dynamic trustiness authentication 
experiment in the safe environment, it mainly 
examines whether the framework can authenticate the 
normal process behavior or not; the other two test is 
aiming at the two typical dynamic intrusion methods to 
test the framework's detecting ability of the malicious 
attack. 

Tlic exact number of AM (API function invoking 
Module) and SIP (Sequential Instmction Piece) ia our 
sample program is shown as Table 1; 
Table 1. Number of AM and SIP 

AM Sell-dcllncd llinclion calling module SIP 

_n 13 110 



3.1 Software Dynamic Trustiness 
Autlientication Experiment in Safety 
Environment 

Normal API function invoking sequence from the 
main thread of the sample program is shown as below: 

Table 2. Normal API Function invoking 
sequence 



GetSystemDirectoryA->GetModuleFileNameA->Istrcat-> 

CopyFileA->RegOpenKeyExA->RegSetValueExA-> 

RegCloseKey->GetProcAddress->FreeLibrary-> 

WSAStartup->socket->htons->bind->listen->accept-> 

GetSystemMetrics->GetSystemMetrics->send->recv-> 

Recv->sprintf->strcpy->MessageBoxA->c\osesocket-> 

accept 

The API function invoking sequence accord with 
AM and MRs, which is described as Table 3. 

The API function invoking in italic is the 
overflowing point. The API "recv" receives data from 
another network termiaal, "sprintf ' does the string 



formatting work , "strcpy" is used to copy the string 
and "MessageBoxA" is used to show the string data. 

In normal sense, the monitor system can record the 
invoking sequence completely in real-time and match 
it with SB AC. The experiment result indicates that the 
software can pass dynamic trustiness authentication 
when it follows the normal sequence. 

Table 3. API function invoking sequence 



and AlVIs 

API Function Invoking Sequence Corresponding 

AM 

GetSystemDirectoryA-> AMI 

GetModuleFileNameA->Istrcat-> 

CopyFileA->RegOpenKeyExA-> 

RegSetValueExA->RegCloseKey-> 

GetProcAddress->FreeLibrary-> AM2 

WSAStartup-> AM3 

socket ->htons->bind->listen-> AM4 

accept->GetSystemMetrics-> AM5 

GetSystemMetrics-> 

send->recv->recv->sprintf->strcpy-> AM7 

MessageBoxA-> 

closesocket-> AMIO 

accept AM5 and other 

AMs 



3.2 Buffer Overflowing Detecting Experiment 

We conduct an overflowing attack and record the 
corresponding API invoking sequence. The real API 
function invoking sequence when suffering the 
overflowing attack is shows as Table 4. 

It can be found that when the overflowing attack 
occurs, the system monitor is able to report that there 
happens a lots other API function invoking after the 
calling of "MessageBoxA", and these unexpected APIs 
are indeed called by the shellcode, which can't be 
tmstedby DTAC. 

Table 4. API function invoking sequence 
when suffering the overflowing attack 



GetSysteniDirectoiyA->GetModuleFileNameA->lstrcat-> 



Recv->sprintf->strcpy->MessageBoxA->LoadLibraryA-> 

Sleep->WaitForSingleObjectEx->ioctlsocket->recv-> 

CreateProcessA->TerminateProcess->closesocket-> 

accept 

3.3 Remote Unknown Thread Injection 
Detecting Experiment 

The well-known Trojan program named gray- 
pigeon injects itself as a remote thread into a specified 
process. In this experiment, we configure the gray- 
pigeon server program to make it inject into a process 
which is monitored imder the dynamic authentication 
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system. The experiment result shows that the new 
thread is not able to pass the dynamic trustiness 
authentication. 

The experiment results and comparison, which can 
demonstrate the efficacy of the dynamic trustiness 
authentication framework, are shown in Table 5. 
Table 5. Experiments result and comparison 
Intended Safety Buffer Remote 
AMs EnvironmentOverflowing Thread 
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(T) : Is API Function invoking sequence part 
according with the trusted behavior of AM? ("•" means 
"Yes", "x" means "No", " \ " means that the behavior of 
software is not trusted) 

(2) : Is the order of AMs according with MRS? 

4. Conclusion 

In this paper, wc propose a dynamic trustiness 
authentication framework based on the integrity of the 
software behavior. This framework carries on the 
analysis to the binary procedure to obtain the Software 
Intended Behaviors Describing Sets. Through 
estimating the API function call sequence which is 
executed by real-time monitoring process whether to 
conform to the software anticipated behavior, the 
framework can examiae the integrity of the software 
behavior, thus authenticate the dynamic trustiness of 
the software. 
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